Empirical analysis in software process simulation modeling

Software process simulation modeling is increasingly being used to address a variety of issues from the strategic management of software development, to supporting process improvements, to software project management training. The scope of software process simulation applications ranges from narrow focused portions of the life cycle to longer-term product evolutionary models with broad organizational impacts. This paper discusses some of the important empirical issues that arise in software process simulation modeling. We ®rst address issues concerning real-world data used to (1) establish input parameters to a software process simulation model, and (2) establish actual organizational results against which the modelÕs results (i.e., outputs) will be compared. On the input side, the challenges include small sample sizes, considerable variability and outliers, lack of desired data, loosely de®ned metrics, and so forth. On the output side, the paper addresses (1) veri®cation and validation of the model, and (2) quantitative approaches to evaluating model outputs in support of managerial decision making including ®nancial performance using Net Present Value (NPV), multi-criteria utility functions, and Data Envelopment Analysis (DEA). The paper focuses on the stochastic modeling using Monte Carlo simulation. The paper is grounded in the authorsÕ practical application experiences, and major points are illuminated by examples drawn from that ®eld work. Ó 2000 Elsevier Science Inc. All rights reserved.